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PREFACE 



This booklet presents an outline 
for the total scheme of evaluation in 
a school district. Evaluation of (1) 
programs, (2) teachers, (3) students, 
and M materials used in the class- 
rooms are discussed with respect to; 

b. data coltzction 

c. dttlAion-fnaklng 

The booklet begins with a sum- 
mary and a chart that will help an 
administrator identify the present 
status of evaluation in his school 
district* Use it to record the eval- 
uation activities completed or under- 
way in your school(s). The chart is 
intended to be suggestive rather than 
exhaustive. The summary is followed 
by an ex^iancition of each step in 
each of the four areas mentioned 
above * 

It is hoped that upon reading 
this booklet, the administrator will 
find the value (s) of using proper 
evaluative approaches* 



I would like to thank all those 
who participated in reading the manu- 
script and offered their helpful sug- 
gestions. 



Philip S. Saif , Director 
Project Evaluation 
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SUMMARY 



* Evaluation should be applied 
to these four areas: pro- 
grams, personnel, students and 
materials . 

* Evaluation should be carefully 
incorporated during planning. 

* Evaluation should be done for 
further decision-making. 

* Evaluation of these four areas 
should be related. 

It is hoped that upon reading 
this booklet, the administrator will 
find the value (s) of using proper 
evaluative approaches. 
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The word "evaluation" has been 
equated with assessment and sometimes 
with judgment. Evaluation could be 
defined as the process of collecting 
data for decision-making. 

There are two types of evaluation: 

1. Evaluation at the end of a 
program or a year: 

"^ummativt zvaZuation" 



2. Evaluation that starts at 
the stage of planning: 

"^ofimativz zvalaatlon" 



Many administrators applied the 
first type — summative-for a long time 
and discovered that it becomes diffi- 
cult, if not impossible, to implement 
corrective action after the program 
has been completed (after the "damage 
has been done") . 

With formative evaluation, mod- 
ifications, or even corrections will 
be possible. 
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In education, there are four 
major areas for evaluation: 

3. StudzntA 

4, fAatzn.ial6 a6zd in thz cia^A' 



I. EVALUATION OF PROGRAMS 

Upon what bases should a school 
district delete a program or propose 
a new one? 

A. Needs Assessment 

Needs assessment is the deter- 
mination of "things you want to do" 
based upon "what you are doing now." 
The administrator should be able to 
identify the following points: 



1. The purpose to be serv'jd. 

2. The target population. 

{OHiat Izvzt/gKadz 6houtd 6e 
zvaiaatzd?) 



3. The level of satisfaction. 

i^^izd by i/ohat ha4 been 
done?) 

U . The alternatives . 

(I^ th^ fie.6atU afiz not 6at' 
i^iattofiyf what otht/i 
aholc.t6 can thz adminiAtfia' 
ton tonildi.fLl) 



B. Planning for Evaluation 

Any administrator has the respon- 
sibility for planning an evaluation 
program. There are some basic ques- 
tions that an administrator should 
ask. 

7. Why 6houtd 1 have, tvatuation? 

2. (>}hat should 6e tvaiuatzd? 

3. Wfie^e ikall I AtaKtf 

4. (Uho will 6e involvtdf 

5. How much \tiill it tohtl 

6» What will I do with thz in- 
ioMndtion? 

Planning looks at the total pic- 
ture rather than fragments* Some- 
times an administrator gets ambitious 



and tries to evaluate everything at 
once and in the shortest possible 
time. This must not be the case. 
Evaluation is not a "one-shot" treat- 
ment; it is a continuous process. 

Planning should involve two major 
questions : 

1. ifJkat a/ie the. goal6? 

2. iUhat afLQ. the. objZctlvz67 



For the sake of clarification, 
the following definitions are stated! 

A goal 16 a continuing pafipo^t 
that^pH.ovid&6 a Aew^e difizc- 
tion thfioagh timz, A goat i6 
gznzfial in 6copz and may be 
thought oi a6 providing a difizc 
tion oH. an aim ion 6choot di6' 
tfLict6 to wo/ife toMaKd, 



An objzctivt i6 a mzaiuKabiz 
dz6iKzd ftZAult to be atcom- 
pti^htd within a ipzci^izd timz 
pzfiiod. It clo6z6 a gap bztmzn 
a pH.&6tnt situation and a dz- 
iifizd situation within a time, 
ifiamz, OK in oth&K mftd6, iut' 
litlA a dziine.d need. 
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The two definitions form the fol- 
lowing sequence leading from system- 
wide goals to student objectives: 

1. School district goals 

2. School program goals 

3. Curriculum objectives 

U. Unit instructional objectives 
5. Instructional objectives 

The following example illustrates 
the sequence. 

A Goal ; Everyone should understand 
man's relationship to his 
environment. 

This is a goal for the society as a 
whole. To translate this goal into 
objectives means making it suitable 
for each level until it is used in 
the classroom. So, the goal mentioned 
above should be broken down into the 
following steps: 

1. District goal : The student will 
understand man in relation to his 
physical environment. 

2« School Program goal : The student 
will understand man as a user 
(consumer) of resources. 



Curricul u m objectives ; The stu- 
dent should identify and explain 
the effects of: 

(a) geography on his life and 
culture . (geography) 

(b) the earth's atmosphere on his 
life and culture. (meterology) 

(c) culture and social mores on 
his life and culture. (anthro- 
pology) 

Unit Objectives ; 

(If (a) is taken from #3 men- 
tioned above); 

i. The student will analyze the 
effects of proximity to 
oceans as a cultural element. 

ii. The student will analyse the 
relationship of agriculture 
to his life. 

iii. The student will identify 
the effects of elevation and 
terrain on various cultural 
units. 

Instructional Objectives : Each of 
the items mentioned under will 
be translated into written state- 
ments to what the student will do* 
(e.g.) The student will list five 
desirable desiderata for a sea- 
port. 



BEST COPY AVAILABLE 



Observe that there is a difference 
between unit objectives and in- 
structional objectives. An in- 
structional objective is what the 
learner will do after instruction 
takes place. 



C . Col lectio n of Data 

In any evaluation, data is a 
very important component. The follow- 
ing questions are to be asked: 

1. {/Jhat data afit mzdzd? 

2. l6 thtKz any pafit oj( tkz coZlzc- 
t&d data that will not be uAtd? 
If yes » such part should not be 
collected . 

3» What ant thz tmthod6 colltC' 
ting data? 

a. Observation techniques 

b. Questionnaires or opinion- 
naires 

c. Tests (criterion-referenced, 
norm-referenced ) 

d. Video tapes 

e. Audio tapes 

f. Check lists 

g. Other methods (you should 
specify) 



4, itJkat Villi you do i/oith data? 

Once data are collected, analysis 
should take place. Analysis 
could be statistical, comparing, 
or even just numbers (as in the 
case of opinionnaires , when one 
says, for example, that 132 agree 
upon this point and 67 disagree). 

5. To whom v)lll you. fizpofit data? 

Data should be available to any- 
one who is interested. Decisions 
should be based upon the data and 
their analysis; e.g., a teacher 
could be a decision-maker con- 
cerning a student upon receiving 
the needed dataj a superintendent 
is a decision-maker concerning 
programs and personnel upon data 
that he receives. There may be 
more than one person to make a 
decision. 



D. Dec is ion-Making 

In order to make intelligent de- 
cision(s), the decision-maker (s) 
should have all related data to the 
matter. The following example may 
help: 
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A superintendent would like to 
make decisions concerning an ed- 
ucational program; whether or 
not he should keep the program. 
In such a situation, the super- 
intendent should ask for the 
following data: 

* ^hat i6 the. natuKZ tht pKo- 

* (tlhat ciKt tkt mztti oi tkz 
pKogfLum on ^tudznUf 

* tkz pfiogfiam U dzte.Ud, 
muld tkU ai^ztt the. educa- 
tion oi the 6tudent6f 

* iomone l6 ^uQaeiitlnQ a naw 
pftogfiamt upon vikat bailh U 
the pKogKam Kejecttd, and on 
uihat ba6i6 l6 the new pnoQKam 
pKopoAedf etc? 



Decision-making is the desired result 
and culmination of the evaluation 
process. It is also the basis for 
further evaluation. 
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II. EVALUATION OF PERSONNEL 



Connecticut Public Act 7U-278 in- 
dicates that continuous evaluation of 
personnel is mandated. In this short 
discussion, the writer would like to 
raise questions which should be dis- 
cussed by administrators who will be 
responsible for personnel evaluation. 

U Can pzuonnti evaluation be 
objective.? Hot/o? 

2. itiho Mill be accepted a6 eval- 
uatoJt (4 ) ? 

3. itlouid the a66eA6ment 0^ a 
teacheK be baAed upon hi^fhe^ 
penio^mance, ok on the 
achievement 0^ 6tudent6? 

4. A^teK evaluation takeA place, 
then what? 

5. a teachen should a6k an 
evaluatoK, "Wha-t will you be 
looking ioK in my cla66?" , 
what will the amwen be? 



The questions mentioned above, 
and many others, could be answered 
if the school district has a system- 
atic way of evaluating its personnel* 
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A systematic personnel evaluation 
must include definite responses to 
these questions! 

1 . {»hat will tkz ttatheA achizvz 
duKing tlil6 tfzaK? 

2. i»kat Mill tht 4ttudtnU 6e 
able, to do upon ^ini6king thz 
ifzaK i^itk thl^ pafiticulafL 

The responses must be determined, 
agreed to, and recorded (in a written 
agreement) between the teacher and 
his/her evaluator(s) » 



i^hat will th z tzatkzn OLthlz\} Z dufiing 
^Kz yz ax? 

A« In the "affective domain " 

The teacher should state two or 
three statements at the beginning of 
the year and agree on them with the 
evaluator. For example, the teacher 
may state: "Thz 6tudznt6 in my zla&A 
will havz bzttzn. attitadz6 tomKdi 
zazk otkzfL," Or, "Upon i^ini^hing thz 
yzaK, thz ^tadznt^ will lovz mth," 
Or, "I'll havz bzttzK zomunization 
with my 6tudznt6*" Such statements 
and many others could be developed* 
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They will serve as a reminder, and at 
the same time provide guidelines to 
the activities of the teacher towards 
realizing the desired influence. In- 
terval examinations of such state- 
ments will be valuable to the teacher. 
For example, if a teacher of math- 
ematics would like to see positive 
attitudes towards mathematics, the 
teacher can apply any of the tools 
that were developed to measure atti- 
tudes towards mathematics. 

The purpose of using such state- 
ments and examinations is to identify 
strengths as well as weaknesses. The 
strengths can then be reinforced, and 
the weaknesses can be improved* 

B. In the "cognitive domain " 

Some administrators are asking 
teachers to write instructional ob- 
jectives for ?Keir courses. This 
should not be the rule. Such instruc- 
tional objectives have been written 
and published. There are thousands 
of them available. i 



•"•Those who are in the CREC area 
can make use of the available re- 
sources in the Center for Evaluation. 
Instructional objectives published by 
lOX, the University of Massachusetts* 
and Westinghouse are available* Other 
objectives that have been developed 
locally are also available} namely, 
Simsbury's elementary math program* 

m 



The teacher can choose whatever 
is suitable for his/her course. In- 
service training need not exceed the 
level of enabling a teacher to modify 
an existing instructional objective, 
should a completely appropriate one 
not be available. 

If instructional objectives are 
chosen and arranged sequentially, the 
teacher and the evaluator will be 
able to determine the outcome of 
teaching in the cognitive domain. 
How? Each objective includes its 
criterion. Such criteria form tests 
that are known as Criterion-Referenced 
Tescs.^ The students' progress is 
assessed by testing what he / she 
learned. 

Considering these two major 
points (statements for the affective 
domain and instructional objectives), 
much of the evaluator 's work will be 
reduced. The teacher can examine re- 
sults without too much effort. In the 
case of instructional objectives, the 



Further discussion of Criterion- 
Referenced Tests is found in the next 
section of this booklet discussing 
student evaluation. 
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teacher can tell what a particular 
student achieved. In the affective 
domain, scales, questionnaires, etc., 
will be good indicators as to student 
progress, or the lack of it.^ Con- 
sequently, the teacher can identify 
the strengths and weaknesses of the 
student. The evaluator's functions 
should be to help the teacher during 
the year to deal with problems, if 
any, and /or reinforce strengths. By 
doing this, evaluation will con- 
tribute to the educational growth of 
both the teacher and the evaluator. 
The evaluator will gain more know- 
ledge of dealing with new situations, 
and the teacher will gain from the 
experience of the evaluator. There 
is no one way for evaluation to be 
incorporated J each school district 
and each school should set its own 
system for evaluation, keeping in 
mind that it is an on-going process. 



Collecting Data 



As mentioned before, only the 
collected data will be the^ needed 
professional information. This means 
that personal data should not be 
gathered unless there is a real need. 
In collecting data for evaluation of 
a teacher, the following information 
could be considered: 
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* The relationship between the 
teacher and students. 

* The relationship between the 
teacher and other teachers. 

* Methods used in teaching. 

* Outcome of teaching. 

* Scholarship of the teachers. 

* The teacher and his/her im- 
provement based on further 
studies, conferences, in- 
service training, etc. 

* The ability of each student 
and the difference (if any) 
between his/her ability and 
present performance. 



Decision-Making 

Making a decision about a teacher 
MUST NOT be limited to matters of 
hiring and firing. Decisions could 
include points such as: 

/. ifiktJit can thlA ttathzK 6e 

2. Wkat OJLZ th<L aKta^ that 
could be. impH.o\/zd? 
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3. a;fie/te did [ok dA,dn*t) change, 
take place? 

4» Could thl6 tcache.fL be oi 
nofLC value heUhe undzK- 
takcA ^uKthtH. itudiei? 

5. thcKZ i6 a makm66, how 

can the teacher be, he.lpe.dt 



III. EVALUATION OF STUDENTS 



How can a teacher evaluate the 
work of a student? The usual answer 
is: test the student. Until the 
sixties of this century, standardized 
tests were the only way for assess- 
ing students; they are known now as 
Norm-Referenced Tests (NRTs). 

1. The NRT scores of an indi- 
vidual are interpreted by 
relating them to test scores 
of other individuals. 

2. An NRT measures the relative 
standing of an individual in 
a norm group or population. 
(One says that Suzy can do 
more than Sam. But this 
does not answer the question 
of what Suzy can or cannot 
do.) 



18 



The Criterion-Referenced Tests 
(CRTs) shift the emphasis upon norms 
to item proficiency. 

1* A CRT is deliberately con- 
structed to yield measure- 
ments that are directly in- 
terpretable in terms of 
specified performance stand- 
ards. 

2. A CRT answers the question 
of what kind of performance 
can or cannot be demon- 
strated by an individual 
student . 

3. A CRT contains items that 
are tied to specific instruc- 
tional objectives and can be 
grouped under specified 
skill levels. 



The emphasis of a CRT is on item 
proficiency, not on norms. 

The following chart compares 
NRTs and CRTs: 
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NRT 

Group Curriculum 

Proves 

Describes 

References Scores 
to a Norm 

Timed 

Survey Instru- 
ment 

Scores Based on 
Measures of Cen- 
tral Tendency 



Addresses Areas 
of { 

Placement 
Research 

Curriculum Man- 
agement 

Resource Allo- 
cation 

Guidance 



CRT 

Individualized In- 
struction 

Improves 

Prescribes 

References Scores 
To Criteria 

Not Timed 

Comprehensive Exam- 
ination of a Dis- 
cipline 

Results are User 
Oriented in Terms 
of Item Proficiency 
by Skill Level 

Addresses Areas of: 



Placement 

Curriculum Man- 
agement 

Resource Alloca- 
tion 

Instruction 



CRT items are published by th« 
same agencies which publish instruc- 
tional objectives. There are also 
commercial companies publishing CRT 
items. 

Those who are involved in pre- 
paring CRTs expect norms to be estab- 
lished, enabling schools to compare 
local results to a wider base. This 
point is being stressed nowadays be- 
cause school districts are afraid of 
being accused of developing CRTs that 
test trivialities. However, such 
norms cannot be set without active 
participation from the schools. 

Up to this point, the concern 
has been with the "cognitive domain." 
What about the "affective domain"? 

Various methods have been de- 
veloped to evaluate the students' 
feelings, attitudes, values and in- 
terests. Among these methods are: 
scales of attitudes, questionnaires, 
check-lists, video tapes, audio tapes 
discussion groups, etc. The affec- 
tive domain is more difficult to 
measure, but it is not impossible. 
Such tools will help the teacher and 
the school to identify where the 
strengths as well as the weaknesses 
are in the classroom and the school. 
It should not stop at this point, 
however. What 'ihould the teacher do 
about it? What should the school do? 
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Within the past few years, Dr. 
Wells Kively of the University of 
Minnesota started combining the cog- 
nitive and the affective objectives 
and he called this "domain-refer- 
enced. "3 

The evaluation of a student will 
not be complete without the third 
dimension; "psychomotor domain." The 
physical development of a student is 
of equal importance as the intellec- 
tual and the emotional (co?,nitive and 
affective) growth. Fortunately there 
are many "standardized" figures 
against which each student could be 
tested. Once again, the importance 
is not to have every student in a 
particular class perform something, 
but to have each student develop 
according to his/her physical abil- 
ities. For example, one does not 
expect every student to run a mile 
ir U minutes. The psychomotor de- 
velopment is a CRT becaui}e each stu- 
dent can observe the development and 
the differences between what he/she 
did last week, or last month, and 
what he/she can do today. 



Those who are interested In 
this point are referred to CSE mono- 
graph HI mentioned in the biblio- 
graphy. Dr. Hively started this 
movement in 1968. 
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Collection of Data 



What kind of data should the 
school have about a student? The 
e valuator (most probably the teacher) 
should decide on the needed informa- 
tion. If evaluation is in relation 
to the degree of progress in any edu- 
cational area, the evaluator should 
determine whether or not certain data 
are needed. For example, does the 
evaluator need to know the student's 
I.Q.? Socio-economic background? 
Age? Health? More importantly, the 
teacher should know where a student 
is now (level of performance) and 
where that student should be (level 
of ability). Is there any difference 
between the two levels? The follow- 
ing illustrates this idea: 

A school has a group of students 
in the 10th grade, but they read at 
the 7th grade level. The school 
(most probably) assigns a teacher to 
teach these students reading skills. 
The information given to the teacher 
is that the students read at the 7th 
grade level. This means that the 
teacher received achievement data 
rather than ability data. The teacher 
aid not receive any information rela- 
tive to what each student can do* 

Another kind of information that 
is needed, but not usually provided, 



23 



ERIC 



is the level of retention. Does the 
teacher know how long a student can 
retain knowledge? Can the teacher 
predict the achievement of a student 
based upon the provided data? There 
are techniques by which such informa- 
tion could be provided. 



Decision-Making 

The analysis of collected data 
will help the teacher, as evaluator, 
to make an intelligent decision. The 
more information that is provided, 
the better judgment the teacher can 
make. Once again, the idea of evalu- 
ation is not to be confined here to 
promoting or flunking a student. The 
teacher should be concerned with 
questions such as: 

U Hotu much dotl thiA 6tadznt 

2. How much tan thU Atadznt 
itafinl 

3. Hovi much dot^ thlh 6tadtnt 
ntcd to tta^nf 

4. Hou) mch do(L6 thU itadtnt 

Also, by obtaining such information , 
the t eacher will be able to Iden'Bl'y 
v7hatlie7she is aocountabie for, and 
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what he/she i s supposed to do . Th i s 
will help tliie^" teacher to reinforce 
strong learning experiences and help 
the student with his weaknesses. 

If the teacher has information 
to answer these questions, then one 
can truly say that evaluation of a 
student is on the right track. 



IV. EVALUATION OF MATERIALS 



Upon what basis does a school 
district choose a series of books, a 
film, a record, a film-strip, etc? 
Such concern led to the establishment 
of organizations like EPIE (Educa- 
tional Products Information Exchange). 
At present, EPIE is trying to reach 
as many states as possible to train 
personnel in evaluation of materials 
used in classrooms. EPIE is now using 
a format . for evaluation of mater- 
ials. Such a tool, although not per- 
fect, is a step towards helping 
school districts to make sound assess- 
ments of what can be, and what should 
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not be used in classrooms. 



Collecting Data 

What data does the evaluator 
need about the material ( s ) ? Does the 
material contain definite instruc- 
tional objectives? Does it follow 
a systematic methodology in presenta- 
tion? Does it contain criteria for 
its evaluation? Is it relevant to 
the mental age of the students? Is 
it specific? 



Decision-Making 

Making a decision about mater- 
ials used should not be confused with 
the value of the material itself. For 
example, materials explaining the 
metric system could be excellent in 
themselves; but they may not be rel- 
evant to the age level or the 



EPIE conducted a five-day 
workshop for CREC to train school 
district personnel in using the de- 
veloped tool. CREC, now, can help 
the school districts in evaluating 
new materials and/or the materials 
you use now. You may contact CREC to 
help you with the selection of mater- 
ials used in classrooms. 
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sequence of instruction that a school 
district might have. 



CONCLUSION 



After an administrator identi- 
fies the need, and what is to be 
evaluated, and after the evaluation 
takes place, the question is, "Thtn 

Unless a decision is made based 
upon the collected data, evaluation 
would not complete the circle. For 
example, if an existing educational 
program is evaluated, what decision(s) 
will be made? The alternatives fac- 
ing the decision-maker(s) will 
include : 

a. keep the program as it is. 

b. keep the program with some 
modification. 

c* stop the program. 

d. combine this program with 
another one. 

e. make use of this program 
on another level or grade. 

27 



Similar decisions could be made 
with other areas of evaluation. For 
example, a teacher may be a great 
success with 8th graders, but a com- 
plete failure with 12th graders. 

In conclusion, evaluation should 
not be a threatening word. It must 
be a process to which educators are 
attracted because of its benefits. 



ERIC 
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